It is known in the field of robotics that the position of an object in three-dimensional space can be determined from the multiple images produced by a stereo camera system. Such a system typically includes a plurality of image pick-up devices (e.g. video cameras), each of which may be rotated relative to three coordinates axes (the X, Y, and Z axes) by means of appropriate motors and drive control circuits. In addition, each image pick-up device is provided with a zoom lens operated by a motor and drive control circuit, to achieve controllable, variable magnification. In operation, plural image pick-up devices are positioned in a known relative orientation, and the position of an object in three-dimensional space is determined from the images of that object provided by the different image pick-up devices, using a method of triangulation.
To measure the positions of objects in a three-dimensional space from picked-up images of the objects, the correct positions and the configurations of the image pick-up devices are required.
A conventional approach uses points in an image, for which the positions in three-dimensional space are known.
For example, refer to the paper "Camera Calibration for 3D Computer Vision Proc. Int. Workshop on Industrial Applications of Machine Vision and Machine Intelligence" by O. D. Faugeras, G. Toscani, et al.
According to this paper, point marks on a flat plate for which the three-dimensional positions are known are installed in front of the image pick-up device, and the position and the rotational orientation of the image pick-up device are measured, based on the position on the picked-up images of the point marks.
Here, to simplify description, it is assumed that only the rotation and the magnification of the image pick-up devices are unknown, and other parameters such as the position are all known.
Coordinates of a point p in a three-dimensional space are taken as (x,y,z), and a projected point of the point p on an image is represented by P(X,Y). Here, for the sake of simplification, the image pick-up device is assumed to be positioned at an origin (0,0,0) in three-dimensional space. Also, the focal length of the image pick-up devices is taken as f, and the rotations around the x-axis, y-axis and z-axis are taken as .theta.,.phi. and .alpha., respectively. At this time, taking the point transformed by magnification and rotation in three-dimensional space as p'(x',y',z'), the relationship among the values of the coordinates of the point p and the point P are expressed by the following Equations (1) and (2). ##EQU2##
Here, Rx, Ry, and Rz are matrices representing rotation, being given by the following Equations (3), respectively. ##EQU3##
Also, E designates a matrix representing the magnification of the image and is given by the following Equation (3'). ##EQU4##
Next, the positions of n (three or more) points are determined in the three dimensional space. The positions of these n points in the three-dimensional space are represented by p.sub.1 (x.sub.1, y.sub.1, z.sub.1), p.sub.2 (x.sub.2, y.sub.2, z.sub.2) . . . p.sub.n (x.sub.n, y.sub.n,z.sub.n), and the coordinates of those points on a picked-up image are represented by P.sub.1 (X.sub.1, Y.sub.1), P.sub.2 (X.sub.2, Y.sub.2) . . . P.sub.n (X.sub.n, Y.sub.n), respectively.
Here, taking points transformed by rotation in Equations (1) and (2) as P.sub.1 '(X.sub.1 ', Y.sub.1 '), P.sub.2 '(X.sub.2 ', Y.sub.2 ') . . . P.sub.n '(X.sub.n ', Y.sub.n ',) respectively, the following Equation (4) holds. ##EQU5## where, 1.ltoreq.i.ltoreq.n
When each of the rotation angles .theta., .phi. and .alpha. is evaluated correctly, the following Equations (4') hold for all of i. EQU X.sub.i =X'.sub.i EQU Y.sub.i =Y'.sub.i (4')
However, perfect matching is not obtained due to errors in measurement, and therefore the rotation .theta., .phi. and .alpha. are evaluated by the least squares method in such a manner that the value of the following Equation (5) is minimized. ##EQU6##
In such a conventional approach, a real known object must be installed at a known position in the three-dimensional space. However, for an apparatus which moves during operation or an apparatus which often changes an object to be observed, the image pick-up device is required to be moved frequently. Then, the conventional approach necessitates that image pick-up be performed by installing the known object at the known position every time the image pick-up device is moved.
However, it is troublesome and difficult in practice to install the known object at the known point every time the device is moved. It is particularly difficult to use it for a mobil robot, which is a high-speed mobile apparatus in an unmanned environment or in the open air.
The present invention has been achieved in the light of such circumstances, and proposes to provide an image processing method and an apparatus therefor which do not necessitate installing a known object at a known position, and which can detect the rotations and the focal lengths (magnifications) of image pick-up devices by picking-up an object using only a plurality of image pick-up devices, even if the position and configuration of the object are not known at all.
Therefore, the method and the apparatus in accordance with the present invention pick-up images of an object whose position and configuration are unknown, by means of a plurality of image pick-up devices located at different positions, determine the values of coordinates of the object in each image, and estimate the rotations and the focal lengths (i.e. the magnifications of the images of the image pick-up devices). Thereby, basically, the rotations and the focal lengths (magnifications) of the image pick-up devices themselves can be detected by using only the images picked-up by a plurality of image pick-up devices and, therefore, unlike the conventional approach, no calibration is required using an object whose position or configuration in a three-dimensional space is fixed.
The above and further objects and features of the invention will more fully be apparent from the following detailed description with accompanying drawings.